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The Transmission Control Protocol (TCP) is a Transport Protocol 
used in the Internet. In [8], a more general class of candidate Trans- 
port Protocols called "protocols in the TCP Paradigm" is introduced. 
The long run objective of studying this class is to find protocols with 
promising performance characteristics. This paper studies Markov 
chain models derived from protocols in the TCP Paradigm. 

Protocols in the TCP Paradigm, as TCP, protect the network 
from congestion by reducing the "Congestion Window" (the amount 
of data allowed to be sent but not yet acknowledged) when there 
is packet loss or packet marking, and increasing it when there is no 
loss. When loss of different packets are assumed to be independent 
events and the probability p of loss is assumed to be constant, the 
protocol gives rise to a Markov chain {VK„}, where Wn is the size of 
the congestion window after the transmission of the n-th packet. 

For a wide class of such Markov chains, we prove weak convergence 
results, after appropriate rescaling of time and space, as p — > 0. The 
limiting processes are defined by stochastic differential equations. De- 
pending on certain parameter values, the stochastic differential equa- 
tion can define an Ornstein-Uhlenbeck process or can be driven by a 
Poisson process. 



1. Introduction. The Congestion Avoidance algorithm of TCP is de- 
signed to prevent network congestion during the transmission of data over 
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a computer network. It does this by controlling the congestion window, i.e. 
the amount of data "transmitted but not yet acknowledged" by a sender. 
What follows is a simplified description of a more general class of Transport 
Protocols. 

Under appropriate units, the congestion window W determines the maxi- 
mum amount of data that a source can send without acknowledgement. The 
"TCP Paradigm" (see [8]) is a class of protocols that includes TCP (and 
other Transport Protocols). For each protocol in the TCP Paradigm there 
are two functions, mcr(-) and decr{-). If, while the congestion window equals 
W, a packet is found to be lost (or marked, under ECN - see [2] and [12]), 
then the congestion window is reduced by decr{W). However, the congestion 
window is never reduced below some fixed minimum value £ > 0. If there are 
no lost packets, then the congestion window is increased by incr(W). For 
protocols in the TCP Paradigm, incr (W) = ciW and decr{W) = caVF^. In 
the special case of TCP, we have ci = 1, a = —1, C2 = 1/2, and (3 = 1. An- 
other special case of interest is when a = and /3 = 1. This is the algorithm 
which Tom Kelly calls "Scalable TCP" in and [4]. 

Let Wn denote the size of the congestion window after the transmission 
of the n-th packet, and let Xn be the indicator function of the event that 
the n-th packet is lost. We shall assume that the Xn's are independent and 
identically distributed. In particular, we are assuming that p = P{xn = 1) 
is a constant that does not change with time. Under these assumptions, we 
are led to the parameterized family of Markov processes 

(1.1) Wp,n+1 = {Wp,n + ClWp^,(l - Xp,n+l) " Cal^pVp.n+l) V t 

The assumptions we place on the various parameters in the model are: 

(1.2) {Xp,n}$?Li is an iid sequence of {0, l}-valued random variables, 

(1.3) p = P(xp,n = l), 

(1.4) ci > and C2 > 0, 

(1.5) - oo < a < /3 < 1 and ^ > 0, 

(1.6) if /? = 1, then C2 < 1, and 

(1.7) if /? < 1, then i > 0. 

We will frequently drop the dependence on p from our notation and simply 
refer to the processes {xn} and {Wri}- 

We are interested in studying the asymptotic behavior of {W^} as p ^ 0. 
To this end, we define the continuous time process 



(1.8) ■Zp(t) = P^W^ip-iy^ , where 7 = (/3 — a) and v = {1 — 0)7. 



ASYMPTOTICS OF CONGESTION AVOIDANCE 3 

In the case that /? = 1, we wiU show that Zp converges weakly as p — > to 
the process Z defined by 

(1.9) Z{1) = Z(0) + ci r Z(s)" ds - C2 r Z(s-) dN{s), 

Jo Jo 

where is a unit rate Poisson process, independent of Z{0) = limZp(O). 
(Note that this is the conjecture given on page 362 of [8].) We will also show 
that, when ^ > 0, the stationary distributions of the discrete time Markov 
chains {p'''VFn} converge weakly to the unique stationary distribution of Z. 
Questions about the convergence of the stationary distributions when (3 = 1, 
as well as the rate of convergence, are addressed in [9] using techniques that 
differ from those used in this paper. 

In the case that /3 < 1, we will show that Zp converges to the process C 
defined by 

(1.10) at) = c(o) + AciC(s)" - C2c{s)^) ds, 

Jo 

where C(0) = limZp(O). With the exception of the initial condition, the 
process ^ is entirely deterministic. The convergence of Zp to C is therefore a 
law of large numbers type of result. Hence, in the case /? < 1, we can extend 
our analysis and study the fluctuations of Zp around this central tendency. 
Unfortunately, it will not suffice to center Zp by C- We must rather define 

(1.11) Cp(t) = Cp(o)+ [\ci{i-p)Cp{sr-c2Cpisf)ds, 

Jo 

where Cp(0) C(0), and consider the processes 

(1.12) Cp{t) = p-^{Zp{t) - Cp{t)), where t = {u - l)/2. 

We will show that converges weakly as p — > to the process ^ defined by 

(1.13) ;° 

-C2 / a-sfdB{s), 
Jo 

where i? is a Brownian motion and .^(0) = lim^p(O). 

A special case of this last result is worth mentioning. For each p G [0, 1), 
define 



(1.14) 



Cp = (Cl(l -P)/C2) 
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so that Cp(^) = Cp is an invariant solution to (1.11). Also, C(O) = limCp(O) = 
Co is an invariant solution to (1.10). Hence, for an appropriate choice of 
Zp{0), converges to the Ornstein-Uhlenbeck process defined by 

(1.15) d£, = -n^dt + adW, 

where W = —B, 

^ = C2/3(ci/c2)^(^-^) - cia(ci/c2)^("-^) 

and 

(Note that this is the conjecture given on page 364 of [8].) We will also 
show that the stationary distributions of the discrete time Markov chains 
{p~'^ {p^Wn — Cp)} converge weakly to the unique stationary distribution of 
the above Ornstein-Uhlenbeck process. 

2. Main Results. We first consider the case /3 = 1 and begin by cata- 
loging some properties of the limit process Z. 

Lemma 2.1. // ^(0) > a.s., then the stochastic differential equation 
(1.9) has a unique solution Z . With probability one, Z{t) > for all t > 0. 
Moreover, if t = inf{t > : Z{t) = cq}, where cq is given by (1.14), then 
T < oo a.s. 

Proof. For each realization of the Poisson process, (1.9) can be solved de- 
terministically and the solution is unique. Let 

T = mi{t > : Z{t) ^ (0,oo)}. 

Since Z decreases only at the jump times of the Poisson process, and, with 
probability one, these jump times have no accumulation points, it follows 
that T = oo a.s. 

To show that t < oo a.s., it will suffice to assume that Z(0) = x > 
is deterministic. We first consider the case x < cq. Suppose t{uj) = oo. 
Then Z{t, uj) < cq for all t > 0. Find u > r such that u — r > ^c^'^ and 
N(u,u>) = N{r,uj). Then for all t £ {r,u], 



Z{t,uj) = Z{r,uj) + ci f Z{s,uj)"'ds. 

Jr 
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Since the solution to this integral equation is unique, 

Z{t,Lo) = (ci(l - a)it -r)+ Z(r,u;)^-")^. 

Therefore, 

Co > Z{u,Lo) > (ci(l — a){u — r)y > cq, 

a contradiction. Hence, r < oo a.s. 

We next consider the case x > cq. Define 

o"! = inf{t > : Z{t) < cq} and (T2 = inf{t > ai : Z{t) = cq}, 

so that r < (72, and it will suffice to show that a2 < 00 a.s. Fix L > x 
and define p = inf{t > : Z{t) ^ [co,L]}. Suppose p{uj) = 00. Then 
Z{t,uj) G [co,L] for ah t > 0. Let 

K = mf{u" : Co < u < L} > 0. 

Find u > r such that u — r > {L — cq)/{ciK) and N{u,uj) = N{r,uj). Then 

ru 

L > Z{u,Lo) = Z{r,uj) +ci / Z{s,uj)°'ds > cq + ci{u - r)K > L, 

Jr 

a contradiction. Hence, p < 00 a.s. 
Now, observe that 

Z{tAp) = x+ / (ciZ(s)" - C2Z{s)) ds-C2 / Z{s-) dM{s), 
Jo Jo 

where M{t) = N{t) — t is the compensated Poisson process. If s < t A p, 
then Z{s) > cq = {01/02)'^ ■ This implies that ciZ{s)°' — C2Z{s) < 0. Since M 
is a martingale, E[Z{t A p)] < x. Letting t 00 gives < x. Hence, 

P{Z[p) = L) < x/L. Note that either Z{p) = L or Z[p) < cq. Therefore, 

= 00) < P{Z{p) = L)< x/L. 

Letting L — > 00 shows cti < 00 a.s. 

As in Theorem V.6.35 in [11], Z is a strong Markov process. Therefore, 

P{a2 = 00) = E[P^^''^\t = 00)]. 

But Z{(Ti) < Co, and we have already shown that P^{t = 00) = for all 
X < Cq. Hence, (T2 < 00 a.s. □ 
We are now prepared to state our main results for the case /? = 1. If /Up 
and fi are Borel measures on a metric space S, then the notation fip ^ fi will 
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mean that fip converges weakly to fi as p ^ 0, that is, Jg f dfip — > Jg f dfi 
as p — > for all bounded, continuous / : S ^ M. If Xp and X are 5'-valued 
random variables, then Xp => X will mean that PXp^ PX~^ . When Xp 
and X are processes, we will take our metric space to be Dj^dfO, oo), the 
space of cadlag functions from [0, oo) to W^, with the Skorohod metric. See 
[1] for details. 

Theorem 2.2. Suppose (3 = 1. Let the processes Zp he given by (1.8) 
and suppose that Zp{0) =^ Z{0), where Z[0) > a.s. Let Z be the unique 
solution to (1.9). Then Zp =^ Z. 

Theorem 2.3. Suppose (3=1 and ^ > 0. Then the Markov chain {Wn} 
has a unique stationary distribution. Moreover, the process Z given by (1.9) 
has a unique stationary distribution r] on (0,oo). For each p > 0, let r]p be 
the stationary distribution for the Markov chain {p'^Wn}. Then rjp =^ r/. 

For some results on stationary distributions in the case (3=1 and i = 0, 
see [10]. 

For the case P < 1, we need some preliminary definitions. Assume that 
for all p G (0, 1), the processes {Wp,n} are defined on the same probability 
space {Q,,J^,P). Define the a-algebra 

(2.1) To = aiWp,o:0<p<l)\jM, 

where TV denotes the collection of events D ^ with P{D) = 0. 

Theorem 2.4. Suppose (3 < 1. Let the processes Zp be given by (1.8). 
Suppose that Zp{0) =^ C(0)) where C(0) > a.s. Let ( the unique solution to 
(1.10). Then Zp =^ Moreover, if Zp{0) C(0) in probability, then Zp ^ 
in probability. 

Theorem 2.5. Suppose (3 < 1. Let the processes Zp be given by (1.8). 
For each p S (0,1), let Cp(0) be a strictly positive random variable defined 
on {Q,J^,P). Assume that Cp(0) is J-Q-measurable and Zp{Q) — Cp(0) — > in 
probability. Define Cp and ^p by (1.11) and (1.12), respectively. 

Suppose that there exists a pair of random variables (^(0),C(0)), defined 
on {Q,,(F,P), such that C(0) > a.s., Cp(0) ~^ C(0) in probability, and 
('?p(0)' Cp(0)) ^ ('^(0)) C(0))- Let B be a standard Brownian motion indepen- 
dent o/ (^(0), C(0)) and define the processes Q and ^ by (1.10) and (1.13), 
respectively. Then (CpXp) ^ i^X)- 
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Theorem 2.6. Suppose /3 < 1. Then the Markov chain {Wn} has a 
unique stationary distribution. For each p > 0, let rjp be the stationary dis- 
tribution for the Markov chain {p~^{p'^Wn — Cp)}. Then rjp =^ rj, where rj is 
the stationary distribution of the Ornstein-Uhlenbeck process given by (1.15). 

3. General Definitions. Define 

An = (^ - Wn-1 - ClT^°„i(l - Xn) + C2W^-lXn) V 0, 

SO that 

Wn+l = Wn + CiW^ - {ciW: + C2W^)Xn+l + A„+i. 

If we let W{t) = VF[tj , then we can rewrite this recursive relation as the 
integral equation 

W{t) = W{0) + ci /* VF(s-)" dm{s) 
Jo 



l\ciW{s-T + C2W{s-f) dS{s) + L(i), 
Jo 



where 

m{t) = [t\, S{t) = Y.Xj, and L{t) = ^ A,-. 

j=i j=i 

Using (1.8), it is then easy to see that 

Zp{t) = Zp{0) + ci r Zp{s-r diTipis) 
(3.1) ^ J 

-cip( Zpis-r dSp{s) - C2 [ Zp{s-fdSp{s) + Lp{t), 
Jo Jo 

where 

mp{t) =p''m{tp-''), Sp{t) =p''-^S{tp-''), and Lp{t) =p^L{tp-''). 
Note that if we define the filtration 

then nip, Sp, and Lp are all {J^fj-adapted. 

Define the M^-valued cadlag {.Ff }-semimartingale 

Yp = (nip, Sp) 
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and define the function Gj, : — > M by 

Gp{x) = (Cix", -Cipx" - C2X^)l{a,>o}- 

Tlien (3.1) becomes 

Zp(t) = Zp(0) + f Gp{Zp{s-)) dYp{s) + Lp{t). 
Jo 

To show that Zp converges as p ^ 0, we will apply the theorems in [•'">]. 
This approach, however, comes with two technical difficulties. The first is 
the presence of the local time term L^; the second is the fact that Gp may 
have a singularity at the origin. To deal with these issues, we introduce the 
process Zp, defined as the unique solution to 

(3.2) Z;{t) = Zp{0) + Gl{Z;{s-)) dYp{s), 

where Gp = Gp(e)l(_oo,£) +G'pl[e,oo)- To quantify the sense in which Zp and 
Zp are close, we define the functional : D^d[0, oo) — > [0, oo] by 

he{x) = inf{t > : \x{t)\ A \x{t-)\ < e}, 

and the stopping times rp(e) = he{Zp), and we observe that 

(3.3) Lp = and Zp = Z^ on [0, Tp{e V p^i)). 

By (3.5.2) in [1], if two cadlag functions x and y agree on the interval [0, t), 
then d{x,y) < e~*, where d is the metric on D]jjd[0, oo). 

4. Convergence of Zp. In this section, we will prove Theorems 2.2 
and 2.4 by applying the theorems in [5] to the processes Zp given by (3.2). 
We must therefore define the processes to which they converge in the cases 
/3 = 1 and /3 < 1. 

Let G{x) = (cix", -C2X^)l{a;>o} and = G(e)l(_oo,e) + Gl[^^^), and 
note that Gp G^ uniformly on compacts as p ^ 0. Let A'' be a unit rate 
Poisson process, define 

Y{t) = {t,N{t)f a.nd y{t) = {t,tf, 

and let Z^ and be the unique solutions to 

(4.1) Z%t) = Z{0) + /* G%Z%s-)) dY{s), 

Jo 

(4.2) C(t) = C{0)+ f G'{C{s-))dy{s), 

Jo 
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where Z{0) and N are independent. Note that if /3 = 1, then = Z on 
[0,hs{Z^)) and hs{Z^) = he{Z) oo a.s. as e ^ 0. Hence, Z) < 

exp(— —> a.s. That is, Z^ ^ Z a.s. in -Dir[0, oo). Similarly, if /3 < 1, 
then = C on [0, /le(C^)), he{C,'^) = /i£(C) ^ oo a.s., and — > C a.s. in 
1?r[0,oo). 

We will show that Z^ ^ Z^ and ^ C^- To pass from this to the 
conclusions of Theorems 2.2 and 2.4, we will need the following lemma, 
which is easily proved using the Prohorov metric. (See Section 3.1 in [1].) 

Lemma 4.1. Let {S,d) be a complete and separable metric space. Let 
{Xp}p>o be a family of S -valued random variables and suppose, for each e, 
there exists a family {Xp}p>o such that 

\\ms\iY>E[d{Xp,Xt)] < 6e, 

where 6s ^ as e — > 0. Suppose also that for each e, there exists such 
that Xp as p — > 0. Then there exists X such that Xp =^ X and 

X. 

Proof of Theorem 2.2. Suppose P = 1, Zp is given by (1.8), and Zp{0) ^ 
Z{0), where Z{0) > a.s. Let Z be the solution to (1.9). 

Let Z^ and Z" be given by (3.2) and (4.1). We first show that Z^ Z^ . 
Recall that — > uniformly on compacts. Also observe that Sp ^ N (see, 
for example. Problem 7.1 in [1]). Hence, since Zp{0) and Yp are independent, 
{Zp{0),Yp) =^ {Z{0),Y) in L>b;3[0,oo). Hence, by Theorem 5.4 in [5], it wih 
suffice to show that Yp has a semimartingale decomposition Yp = Mp + Ap 
into a martingale part and a bounded variation part such that for each t > 0, 

(4.3) sup E[[Mp]t + Tt{Ap)] < oo, 

p 

where [Mp]t is the quadratic variation process of Mp and Tt{Ap) is the total 
variation of Ap on the interval [0, t] . 
For this, define 

Sp(t) = Sp(t) - mp{t) = p"'^ {xj - p), 

i=i 

so that Sp is an {jF^}-martingale. Note that Tt{mp) = mp{t) and 

(4.4) E[Sp]t=p^''-^ E\xj-p\^=P^''~\tp-'^\p{l-p)<tp'^-\ 
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Since /? = 1 implies = 1, this verifies (4.3) and shows that ^ . 

By passing to a subsequence, we can assume there exists a [0, oo]-valued 
random variable (y{e) such that {Zp,h^{Zp)) ^ {Z^,a{e)). By (3.3), 

limsup£^[(i(Zp,Zp)] < limsup ^[exp(-rp(e V p'^£))] 
= limsup £^[exp(— /i£(Zp))] 

p^O 

We claim that E[exp(—a{£))] < E[exp{—h^{Z'^))]. To see this, let us assume 
by the Skorohod Representation Theorem (see Theorem 3.1.8 in [1]) that 
{Zp,he{Zp)) — > (Z^,cj(e)) a.s. Then hi;{Z^) < a{e) a.s., which proves the 
claim. 

Since he{Z^) = he{Z) — oo a.s. as e ^ 0, we can apply Lemma 4.1 to 
conclude that Zp ^ Z. □ 



Proof of Theorem 2.4. Suppose /? < 1, Zp is given by (1.8), and Zp(0) ^ 
C(0), where C(0) > a.s. Let C be the solution to (1.10). 

Note that /3 < 1 implies v > 1. Hence, (4.4) implies that (4.3) is satisfied 
and 5p ^ in probability. Therefore, {Zp{Q),Yp) =^ {Z{0),y) in D^s [0, oo). 
By Theorem 5.4 in [5], Z^ C- By Corollary 5.6 in [o], if Zp(0) ^ C(0) in 
probability, then Zp — > in probability. By the same argument as above, 
this implies that Zp converges to ( in distribution or in probability, respec- 
tively. □ 

5. Fluctuations of Zp. In this section, we prove Theorem 2.5. Let us 
first recall the setting of that theorem. We have (3 < 1 and Zp given by 
(1.8). Recall that the processes Zp are all defined on the same probability 
space (fi, J^, P). For each p > 0, Cp(0) is an .Fo-measurable random variable, 
where To is given by (2.1), such that Cp(0) > a.s. and Zp{0) — Cp(0) 
in probability. The processes Cp and ^p are then given by (1.11) and (1.12). 

To apply the theorems in [5], we wish to write S^p as the solution to a 
stochastic differential equation. By (1.11) and (3.1), we have 

Cpit) = Cp{0) + ci(l - p) f p-^Zp{s-r - CpisT) dmp{s) 

Jo 

(5.1) -C2 [\~^iZp{s-f-Cpisf)dSp{s) 

Jo 

-C2 [ Cp{sfdBp{s) + Rp{t), 
Jo 
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where 

Bp{t) = p-^Sp{t) - mp{t)) = p^-^-^y' Y: {x,-p) 



and 

rt 



Rp{t)=p-^ / {ci{l - p)Cp{sr - C2Cp{s f)d{mp{s) - s) 

(5.2) ^ 

- cip [ Zp{s-TdBp{s) +p~^Lp{t). 
Jo 

Given a real number r, let us define the continuous function Fr : (0, oo)'^ — > R 

by 

Using this, (5.1) becomes 

Ut) = CpW + - P) f\p{s-)V'^{s-)dmp{s) 

(5.3) ^ ^ 

-C2I ip{s-)V^p{s-)dSp{s)-C2 I Cp{sfdBp{s) + Rp{t), 
Jo Jo 

where T>1 = Fr{Zp,Qp). 

Proof of Theorem 2.5. Suppose that there exists a pair of random vari- 
ables (^(0),C(0)), defined on {Vt,T,P), such that C(0) > a.s., Cp(0) C(0) 
in probability, and (^p(O), Cp(0)) =^ (^(0),C(0)). Since the map that takes a 
point x > to the unique solution to (1.11) with Cp(0) = a; is continuous, 
Cp ~^ C ill probability and ('^p(O), Cp) ^ (■^(0), C)- Also, since F,. is continuous, 
Pp rC,{-y~^ in probability. 



p 
Let 



Up{t) = ?p(0) - C2 f U^f dBp{s) + i?p(t), and 
Jo 

yp{t) = ci{l - p) ( V^{s-)dmp{s)-C2 I V^{s-)dSp{s), 



so that (5.3) becomes 



(5.4) ut)=^pit)+ / Cp{s-)dyp{s). 







We will apply the theorems in [5] to this integral equation. 
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We first show that i?p ^ in probability. By the Martingale Central Limit 
Theorem (Theorem 7.1.4 in [J]), Bp =^ B, where B is a standard Brownian 
motion; by Theorem 2.4, — > ^ in probability; and by (4.4), {Bp} satisfies 
(4.3). Hence, by Theorem 2.2 in [5], 

cip f Zp{s-T dBp{s) ^ 

JO 

in probability. By (3.3), p~'^Lp = on [0, hpji{Zp)). Since hpji{Zp) ^ oo in 
probability, p~'^Lp — > in probability. 

For the final term in (5.2), note that p~'^\mp{t) — t\ < p^""^ and u — 
T = (i/ + l)/2 > 0. Hence, p~'^{mp{t) — t) — uniformly. Let fp{s) = 
ci(l — p)Cp('S)" — C2Cp{s)^- Since (p ^ ( in probability, we can pass to a 
subsequence and assume that Cp C uniformly on [0,t], a.s. By (1.11), this 
implies that C' uniformly on [0, t]. Hence, fp and fp converge uniformly. 
Integrating by parts, we have 

p-^ f fp{s) d{mp{s) -s)= p-^ fp{t){mp{t) - t) 
Jo 

-p"^ {mp{s) - s)fp{s)ds, 
Jo 

which goes to zero uniformly and completes the proof that Rp ^ in prob- 
ability. 

It now follows from Theorem 5.2 in [ ■] that (Up, 3^p, Cp) =^ (^, 3^, C)i where 

U{t) = ^{0) - C2 f\{sfdB{s), and 
Jo 

y{t) = ci[ aC{sT-^ ds - C2 I PCisf^Us, 
Jo Jo 

and i? is a standard Brownian motion independent of (^(0),C(0)). By Re- 
mark 2.5 in [5], we may apply Theorem 5.4 in [o] to (5.4) and conclude that 
(^P) Cp) ^ (■?) 0) where ^ is the unique solution to (1.13). □ 

6. Stationary Distributions. In this section, we prove Theorems 2.3 
and 2.6. For this, we make time continuous in a slightly different manner 
than before. Let be a unit rate Poisson process independent of {Wn} and 
let X{t) = VFjv(t). Then X is a continuous time Markov chain on E = [i, oo) 
with generator 



Aip{x) = p{ip{x - g{x)) - ip{x)) + (1 - p){(p{x + cix") - v?(x)). 
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where g{x) = {c2X^) A {x — £). When /3 = 1, we wiU study the process 

Zp{t)=p'^X{tp-^), 
whereas when P < 1, we will consider 

where Cp is given by (1.14). It is easy to see that a probability measure is a 
stationary distribution for {p'^Wn} or {p~'^{p'~'Wn — Cp)} if and only if it is 
a stationary distribution for Zp or ^p, respectively. 

Lemma 6.1. If ^ > 0, then {Wn} has a unique stationary distribution. 

Proof. It will suffice to show that X has a unique stationary distribution. 

Let ip{x) = X so that 

Aip{x) = —pg{x) + (1 — p)cix". 

Since g{x) = C2X^ for x sufficiently large, Aip is bounded above and Aip{x) 
— OO as x — > OO. By Lemmas 4.9.5 and 4.9.7 in [1], the family of probability 
measures {iJ,t}t>i defined by 

Mn = ^ /*p-(x(s)€r)ds 

t Jo 

is relatively compact. By Theorem 4.9.3 in [1], any subsequential weak limit 
of {/ij} is a stationary distribution for X. 

To show that the stationary distribution is unique, it will suffice to show 
that for all x £ E, 

T = \ni{t > : X{t) = £} < DO, P^-a.s. 

(See, for example. Problem 4.36 in [1].) Let x € -E be arbitrary and let e > 0. 
Choose M such that /X(([£, M]) > 1 — e for all t > 0. Note that there exists 
K >0 such that py{T < co) > K for all y G [£, M]. 
Define the stopping times tq = and 

Tj+i = inf{t >Tj + l: X{t) < M}, 

and note that tj oo a.s. By the strong Markov property, 

P(r = oo,r,- < oo) = i^[l{,>,^,,^,<oo}P''^"^'^(r = oo)] 

< (1 - K)P{t > Tj,Tj < oo) 
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Letting j — > oo shows that P{{t = 00} D D) = 0, where D is the event that 
Tj < 00 for all j. Note that 

1 /•* 



1 /■* 

lz)c < hmmf- l{x{s)>M}ds. 



Hence, by Fatou's Lemma, P[D^) < liminff_>oo /^^((M, 00)) < e. Therefore, 
P{t = 00) = P{{t = 00} n -D*^) < e. Since e was arbitrary, t < 00 P^-a.s. 
and the stationary distribution is unique. □ 

Proof of Theorem 2.3. In what follows, C and K will denote strictly 
positive, finite constants that do not depend on p and may change value 
from line to line. 

Suppose (3 = 1, £ > 0, and r]p is the stationary distribution for {p'^Wn}- 
Then rjp is the stationary distribution for Zp, which is a continuous time 
Markov chain on Ep = [p'^i, 00) with generator 

Apip{x) = Lp{x - p'^g{p~^x)) - ip{x) 

+ p~^{l - p){ip{x + pcix") - (p{x)). 

Let (p{x) = X + x~^, so that 

Apip{x) = —p^g{p~'^x) + (1 — p)cix°' 



+ 



x{x—p'yg{p "fx)) x{x + pcix°') 
Since x 1— > 1 + pcix°'~^ is decreasing, 

1 + pcix"-^ < 1 +pci(p^^)"-^ = 1 + cir-^ 

for all X G Ep. Hence, 



Ap^ix) < -p-^gip-^x) + Cx" + P^aiP Jx) _ 

x[x — p'g{p 'x)j 



whenever p < 1/2. 

If X > p^i/{l — C2), then g{p~'^x) = C2P~'^x and 

Ap^{x) < -Kx + C7x" + Cx-^ - Kx"-'^. 

If X < p'^i/ (1 — C2), then g{p~^x) = p~^x — £ and 

Ap^{x) < Cx° + ""-Py _ < Cx" + {pH)-^ - Kx"-'^. 

xp'^l 
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But in this case, {p'^i)~^ < Cx~^. It therefore fohows that 

Apip{x) <C -Kx- ifx"-^ 

for all X G Ep. 
Let e > 0. Define 

L = sup sup Apip(x) < oo 

p<l/2 xeEp 

and let in = L{1 — e)/e. Choose M > such that x ^ [M~^,M] implies 
Apip{x) < -m for ah p < 1/2. By Corollary 4.9.8 in [1], 

r]p{[M-\M]) > r]pi{x : Apip{x) > -m}) > = I - e. 

L + m 

The family of measures {r/p} is therefore relatively compact on (0, cxd). By 
passing to a subsequence, we can assume that r]p =^ i] for some probability 
measure i] on (0,oo). 

Now let p'^Wq have distribution i]p and let Zp be given by (1.8). By 
Theorem 2.2, Zp Z, where Z satisfies (1.9) with PZ(0)~^ = r?. Fix 
h < ■■■ <tn- Then 

{Zpih), Zp(t^)) = p^W\^t,p-^ , • • • , VFLi„p-ij ) 

= P^'iWo, M^[t2p-ij-Ltip-'J ' • • • ' ^Linp-iJ"Ltip-'j) 
= {Zp{0), Zp{t2 - ti), . . . , Zp{tn - h)) + e, 

where ej = Zp{hj) — Zp{tj — ti) and hj = {[tjp~^\ — [tip~^\)p. Note that 
hj — > tj — ti as p — i- and, for fixed t, Z is almost surely continuous at t. 
Hence, e — > a.s., which gives 

(Zpih), Zp{tn)) ^ (Z(0), Z{t2 - tl), . . . , Z{tn - h)). 

But 

{Zp{ti),...,Zp{tn)) (Z(ti),...,Z(t„)), 

so Z is a stationary process, and 77 is a stationary distribution for Z. The 
uniqueness of t] follows from Lemma 2.1. □ 
For the proof of Theorem 2.6, note that (,p is a continuous time Markov 
chain on Ep = [p~'^{p^l — Cp), 00) with generator 

f6 1) '^P'^*^^) =p~''^'^{ip{x -p"'~^g{p'^~"'x + p~"'cp)) -ip{x)) 

^ ■ ' +p~''{l-p){ip{x+p-'-^ci{p^-^x + p-^Cpr)-ipix)). 



16 



T. J. OTT AND J. SWANSON 



We will use the same argument as in the proof of Theorem 2.3, this time 
using the Lyapunov function ip{x) = \x\''\ where r is sufficiently large. Our 
key estimate on Api^{x) is given in the following lemma and is valid as long 
as is not too large. 

Lemma 6.2. Suppose /3 < 1. Let f{x) = \x\'^ , where r > 2, and let Ap be 
given by (6.1). Let < 6 < M < oo be arbitrary. Then there exists po > 
and strictly positive, finite constants C and K such that 

Apip{x) <C -K\xY 

for all p < Po and all x Ep satisfying 6 < p'^x + Cp < M . 

Proof. For notational simplicity, let us define yp{x) = p'^x + Cp so that 

Apip{x) = p'"^^ {(fix - p'^"^ gip'^^Vp)) - ip{x)) 

+ p-'^il - p){ip{x + p'-^ci{p-^ypr) - ifix)). 

Either g{x) = C2X^ or g{x) < C2X^. Note that there exists xq > £ such that 
g{x) = C2X^ if and only if x > xq. Hence, if g{p~^yp) < C2{p~'^yp)^, then 
P~^yp < ^Oi which implies x < p~'^{p^XQ — Cp). If p is sufficiently small, this 
implies x < 0. Since (p is decreasing on (— oo,0], it follows that 

Ap^{x) < p-''+H^{x-p^-^-""'c2y^) - ^{x)) 

+ p-^{l-p){y,{x+p^-^-''^ciy^) - y,{x)) 

for all X G Ep. 
Observe that 

\tf{z) -f{x) - (p'{x){z - x)\ = 



z — u){p"{u) du 

<C7|z-xp(|xr'-2 + |zr'-2) 
< C\x\'-^\z - x\^ + C\z - x\ 



Hence, 



Ap^{x) < -^'{x)p-^ip-'+'+^-^f'c2y^p - p-'+^-^'^ciil - p)y^) 

+ C|xr^-2(p--+l+27-2r-27/3^2^2/3 ^ ^-,+2j-2r-2^a ^2y2a^ 
_|_ (J^p-i^+i+r-y-rT-r-yfi^ryrfi _^ p-iy+r-y-rT-r-ya^ryra-^ 
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We can simplify these exponents by observing that 

—V + 7 — 7a = 
-i/ + 1+ 7- 7/3 = 
-1/ + 27 - 2r - 27a = 1 
-1/ + 1 + 27 - 2t - 27/3 = 
— V + 1 + r7 — rr — r^(5 = T{r — 2) 

—v + — rr — vya = r — 1 + T(r — 2). 

Thus, 

Since <^'(x) and C2y!^ — ci(l — p)^^ have the same sign, this gives 



AM^) < -r\xY-h--\c2yl - ci(l - + C\xY-\yf + 



2o\ 



2\ . N / II » I -"/y *,„^l I I IJ ^ up 



for aU X ^ Ep. 



If r > 2 and 6 <yp< M, then 

Apifix) < -r\xr'p--C2y^\y^p-'' - c^-"| + C\xr^ + C. 
By the Mean Value Theorem, 



ipp{x) < -K\x\'' ^\yp - Cp\ + C\x\'' ^ + C 
= -i^|xr + C|xr-2 + C, 

which completes the proof. □ 
The following two lemmas provide the needed estimates on Apip in the 
extreme regimes. 

Lemma 6.3. Suppose /? < 1. Let (p{x) = \x\^' , where r >2, and let Ap be 
given by (6.1). Then there exists pq > 0, M < 00 and K > such that 

Ap^ix) < -Jf|x|(^-i)^('^-i+^) 

for all p < po and all x ^ Ep satisfying p'^x + Cp> M. 
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Proof. Let p < po and yp = p'^x + Cp > M. If po is sufficiently small and M 
is sufficiently large, then x > Kp~'^ and yp < x. By (6.2), 

Ap^ix) < -K\xrVp + C\xY-hf + Cyl" 

= -\xrVp{K - C\x\-^yl - C|x|-^-+iy^(^-i)). 

If /3 < 0, then for p sufficiently small, 

Apifix) < -\x\'-'^y^{K - C\x\-^ - ^1x1"''+^) < -K\x\'^-^+'^ . 

If /3 > 0, then 

Apifix) < -Ixl'-^y^iK - C\xf~^ - (:7|x|(^-i)("-i)), 

so for p sufficiently small, Apip{x) < —K\xY~^y^ < —K\x\'''~^. □ 

Lemma 6.4. Suppose /3 < 1. Let ip{x) = \xY , where r >2, and let Ap be 
given by (6.1). Then there exists po > 0, 6 > and K > such that 

Apifix) < -K\x\'Hr-2a/il-m 

for all p < Po and all x Ep satisfying p'^x + Cp < 6. 

Proof. Let p < po and yp = p'^x + Cp < 6. Note that since x G Ep, yp > p^i. 
If Po a.nd 5 are sufficiently small, then x < and Kp~'^ < \x\ < Cp~'^ . By 
(6.2), for 6 sufficiently small, 

Apifix) < -\x\^y^iK\y^-^ - c^-| - Cip^^y^/-^ + p'^+'y!^) 

_ (J (jp-r+T{r-2) yr(3-a _|_ ^Tr+r-l+r(r-2)yro-o^'j 
< -\xYy;iK - C(p2-y2/3-" +p2r(r-l)yr/3-a^ 
- C7(p2-+ly^" + p(2-+l)(-l)y^(-l))). 

Let us first estimate the termp^^j/p^"". If 213— a > 0, thenp^^y^^"" < Cp^^. 
If 2/? - a < 0, then p^^y^/J-a < Qp2r+^{2p-a) _ ^ote that 2t + 7(2/? - a) = 
7 + 1. Hence, for all values of a and /?, there exists some s > such that 

Similarly, for the remaining terms in the above inequality, we observe that 

2r(r - 1) + 7(r/3 - a) = (2r + 7/3)(r - 1) + 1 = 7(r - 1) + 1 
2r + 1 + 7a = 7 
(2r + l)(r - 1) + 7a(r - 1) = 7(r - 1). 
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Therefore, if po is sufficiently small, then Apip{x) < —K\x\'^yp. If a < 0, 
then Apip{x) < —K\x\'''. If a > 0, then 

Since 'ja/r = 2a/ (1 — /?), this completes the proof. □ 

Proof of Theorem 2.6. Suppose /3 < 1 and ijp is the stationary distribution 
for {p~^{p^Wn — Cp)}. Then r]p is the stationary distribution for ^p. Let 
ip{x) = \x\^' , where r > 2. By Lemmas 6.2, 6.3, and 6.4, if r is sufficiently 
large, there exists po > and strictly positive, finite constants C and K 
such that 

Apip{x) <C-K\x\'' 

for some s > and all p < po and x € Ep. As in the proof of Theorem 2.3, 
this implies that the family of measures {rjp} is relatively compact on M. By 
passing to a subsequence, we can assume that rjp ^ rj for some probability 
measure ry on M. 

Let p~^ip'^Wo—Cp) have distribution rjp, let Zp be given by (1.8), and let 
be given by (1-12) with (p = Cp. Note that ^p(O) converges in distribution, so 
p^^p(O) = Zp{0) - Cp(0) ^ in probability. Hence, by Theorem 2.5, Cp C, 
where C satisfies (1.15) with P^(0)~^ = rj. As in the proof of Theorem 2.3, 
,^ is a stationary process, so r] is the stationary distribution for ^. □ 
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